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Abstract 

This  paper  uses  management  control,  resource-based,  systems-based,  and 
contingency-based  strategy  theories  to  describe  a  large  US  manufacturing  company’s  efforts  to 
improve  profitability  by  designing  and  using  a  performance-measurement  model  (PMM).  This 
PMM  includes  multiple  performance  measures  relevant  to  its  distribution  channel  for  products, 
repair  parts  and  maintenance  services.  The  PMM  is  intended  to  reflect  the  company’s 
understanding  of  performance  relations  among  strategic  resources,  operational  capabilities,  and 
desired  financial  outcomes.  The  PMM  also  reflects  its  intended  distribution  strategy,  the  types  of 
performance  necessary  to  achieve  that  strategy  by  its  distributors,  and  its  desired  financial 
outcomes.  Furthermore,  the  company  uses  the  model  to  evaluate  its  North  American 
distributors  and  intends  to  use  these  evaluations  as  a  partial  basis  for  annual  and  long-term 
rewards.  Thus,  the  PMM  embodies  the  measurable  portion  of  the  firm’s  management  control 
system  of  its  distribution  channel. 

The  study  addresses  four  research  questions:  (1)  Are  measure  attributes  important 
considerations  for  performance  measure  choice?  (2)  Does  the  importance  of  attributes  differ 


t.Q\n*n*niKnvtUM7: 


ACQUISITION  RESEARCH:  THE  FOUNDATION  FOR  INNOVATION 


-  177- 


according  to  firm  strategy?  (3)  Does  the  importance  of  attributes  for  design  and  use  differ 
according  to  firm  strategy?  (4)  Does  a  company  trade-off  some  individual  attributes  for  others? 
The  questions  are  investigated  using  qualitative  and  quantitative  analyses  of  archival 
documents  and  interviews  with  top  managers  and  distributors.  Principal  findings  are  that 
measure  attributes  are  important  considerations  for  choice  and  change  of  performance 
measures;  design  attributes  are  more  important  than  use  attributes;  the  importance  of  attributes 
does  not  appear  to  differ  according  to  strategy;  and  some  individual  attributes  are  traded-off  for 
other  attributes. 

Key  Words:  performance-measurement  model,  management  control,  non-financial 
performance  measures,  strategy 

Introduction 

Improving  performance  measurement  at  key  parts  of  the  value  chain  is  one  of 
management  accounting’s  major  roles.  Valid  performance  measurement  allows  a  firm  to 
effectively  describe  and  implement  strategy,  guide  employee  behavior,  assess  managerial 
effectiveness,  and  provide  the  basis  for  rewards.  Managers  and  researchers  from  diverse 
disciplines  have  sought  to  improve  management  of  the  value  chain  by  building  and  using 
performance  measurement  models  (PMM).  PMM  are  comprehensive  models  of  the  firm  as  a 
system,  which  reflect  organizational  knowledge  of  the  relations  among  various  value-chain 
performance  measures.  Many  organizations  reportedly  have  created  PMM  that  model 
performance  relations  among  key  value-chain  activities  and  valued  outcomes  (e.g.,  the 
balanced  scorecard  of  Kaplan  &  Norton,  1996). 

Consulting  reports,  normative  studies,  and  descriptive  theories  predict  that  these 
comprehensive  models  lead  to  superior  performance.  Magretta  (2002)  argues  that  business 
models  are  essential  to  tying  insights  to  financial  results.  Furthermore,  knowledge-based  and 
systems-based  theories  of  the  firm  hypothesize  that  superior  performance  results  from  systemic 
management  policies,  rather  than  myopic  focus  on  elements  of  the  value  chain  (e.g.,  Huff  & 
Jenkins,  2003;  Morecroft,  2002;  Sanchez  &  Heene,  1996).  Empirical  evidence  supporting 
normative  claims  or  theoretical  hypotheses  is  scant  and  is  usually  in  the  form  of  uncritical  self- 
reports  (e.g.,  Rucci  et  al. ,  1998;  Barabba  et  al.,  2002). 

Systematic  management  requires  a  comprehensive  Management  Control  System 
(MCS),  but  not  all  of  a  MCS  need  be  measurable.  However,  the  portion  that  is  feasibly 
measured  should  be  considered  for  the  PMM;  otherwise,  the  organization  might  lose  valuable 
performance  information.  The  choice  of  performance  measures  is  critical  in  reflecting  the 
organization  as  a  system.  Since  an  organization  is  always  adapting  to  its  environment,  it  must 
be  able  to  change  its  performance  measures  to  reflect  current  conditions.  This  study  describes 
the  determinants  of  a  particular  PMM  and  investigates  the  relatively  unexplored  issue  of  choice 
and  change  of  a  functioning  PMM. 

1. 1  Knowledge-based  and  systems-based  theories  and  PMM 

Theories  that  explain  management  policies  based  on  strategic  resources,  capabilities, 
learning,  and  systems  offer  guidance  and  predictions  for  the  choice  of  performance  measures. 
Recent  strategic  management  literature  has  evolved  the  concept  of  a  firm’s  product  strategy 
beyond  Porter’s  (1985)  depiction  of  managing  the  value  chain  to  achieve  competitive  advantage 
(e.g.,  through  product  cost  leadership  or  differentiation).  Porter’s  work  on  the  importance  of 
strategic  positioning  has  greatly  influenced  later  work  that  seeks  to  explain  how  firms  might  use 
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their  resources  to  attain  strategic  positioning.  Research  that  followed  Porter  explains  how  firms 
reach  and  maintain  the  positions  of  strategic  advantage  that  he  described. 

Barney  (1991)  argues  that  successful  firms  achieve  competitive  advantage  by  acquiring 
and  using  unique  resources  to  build  inimitable  capabilities  that  create  strategic  advantages  (see 
also  Amit  &  Schoemaker,  1993;  Kogut  &  Kulatilaka,  2001).  Organizational  learning  theory  by 
Nonaka  and  Takeuchi  (1995)  and  Senge  (1990)  predicts  that  successful  firms  create  strategic 
advantages  by  learning  dynamically  to  use  their  resources  effectively.  This  learning  is  realized 
through  development  and  deployment  of  the  firm’s  capabilities,  processes,  or  competencies  to 
use  resources  (e.g.,  Prahalad  &  Hamel,  1990).  Morecroft  et  al.  (2002)  hypothesize  that 
successful  firms  manage  strategic  resources  and  capabilities  through  holistic  management 
systems;  that  is,  creating  and  maintaining  strategic  advantages  are  enhanced  by  systemic 
management.  Our  accounting  interpretation  of  current  management  theories  is  that  firms  create 
and  maintain  strategic  advantages  or  positions  by  efficiently  creating,  deploying,  and  using 
performance-based  MCS.  Furthermore,  the  measurable  part  of  the  MCS  should  itself  be 
systemic,  in  the  form  of  a  PMM. 


1.2  Prior  work  on  PMM 

The  DuPont  ROI  formula  is  an  early  and  enduring  PMM  that  disaggregates  financial 
performance  into  manageable  elements  (e.g.,  Zimmerman,  1997,  p.  187).  EVA  is  a  similar, 
more  current  and  complex  approach  to  identifying  the  incremental  contribution  to  shareholder 
wealth  and  the  manageable  elements  of  periodic  income  (e.g.,  Adimando  et  al,  1994). 
Rappaport’s  (1999)  approach  to  building  shareholder  value  recognizes  incentive  effects  of  over¬ 
reliance  on  periodic  financial  results  and  seeks  to  mitigate  disincentives.  Because  all  of  these 
models  focus  primarily  on  financial  outcomes,  they  do  not  qualify  as  systems  models;  that  is, 
they  do  not  model  the  determinants  of  financial  performance  even  within  the  boundaries  of  the 
firm. 


More  comprehensive  PMM  include  Otley’s  (1999)  performance  management  model, 
Ittner  and  Larcker’s  (2001)  value-based  management  model,  Epstein  et  al.’s  (2000)  APL  model, 
Kanji’s  business  scorecard  (Kanji  &  Moura  e  Sa,  2002)  and  the  balanced  scorecard  (BSC) 
(Kaplan  &  Norton,  1996;  2001).  These  models  describe  links  among  business  decisions  and 
outcomes  and  serve  to  guide  strategy  development,  communication,  implementation,  and 
feedback  at  multiple  points  along  the  value  chain.  Because  these  comprehensive  PMM  are 
business  models  reflecting  inputs  and  both  intermediate  and  final  outputs,  they  generally  include 
measures  of  operational,  strategic,  financial  and  non-financial  performance.  These  models  truly 
represent  efforts  to  use  organizational  knowledge  to  model  the  firm  as  a  system  and  implement 
management  control. 

This  study  investigates  whether  management  control,  knowledge-based  and  systems- 
based  theories  of  the  firm  are  descriptive  in  a  particular  case.  Although  a  study  such  as  this 
cannot  generalize  to  the  population  of  firms  using  PMM,  the  findings  of  this  study  can  illustrate 
the  theories  applied  to  this  investigation  (Yin,  1994)  and  can  provide  a  foundation  for  theory 
improvement,  replications,  and  large  sample  tests. 

This  study  addresses  four  research  questions: 

RQ1\  Are  measure  attributes  important  considerations  for  performance  measure  choice? 

RQ2\  Does  the  importance  of  attributes  differ  according  to  firm  strategy? 
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RQ3\  Does  the  importance  of  attributes  for  design  and  use  differ  according  to  firm 
strategy? 

RQ4\  Does  a  company  trade-off  some  individual  attributes  for  others? 

The  questions  are  investigated  both  qualitatively  and  quantitatively  using  (1)  archival 
documents  that  describe  the  company’s  distribution  PMM  and  (2)  interviews  with  top  managers 
to  understand  the  nature  of  the  business  and  the  objectives  and  dynamic  structure  of  the  PMM. 
Finally,  the  paper  reflects  on  the  implications  of  this  study  for  performance  measurement  and 
management  control.  Subsequent  sections  of  this  paper  address  performance  measure 
attributes,  the  research  questions,  the  research  site,  analysis  of  qualitative  and  quantitative 
data,  discussion  of  results,  and  conclusions. 

2.  Performance-Measure  Attributes 

Management  control  theory  argues  that  MCSs,  which  include  PMMs,  are  intended  to 
insure  that  employees  (1)  know  what  is  expected  of  them,  (2)  will  exert  effort  to  do  what  is 
expected,  (3)  are  capable  of  doing  what  is  expected,  and  (4)  accomplish  what  is  expected  (e.g., 
Merchant,  1998).  For  more  than  30  years,  researchers  have  known  that  firms  choose  a  portfolio 
of  controls  and  performance  measures  (e.g.,  Khandwalla,  1972).  However,  subsequent 
research  on  firms’  choices  of  performance  measures  often  has  focused  on  broad  dichotomies  of 
measures,  such  as  financial  vs.  non-financial  measures  and  mechanistic  vs.  organic  controls. 
The  theory  commonly  used  in  that  research  likewise  characterizes  the  contingencies  affecting 
choices  of  measures  and  controls  as  broad  dichotomies  (e.g.,  high  vs.  low  environmental 
uncertainty;  old  vs.  new  technology).1 

One  particularly  popular  research  stream  predicts  that  firms  operating  in  complex  and 
risky  environments  rely  heavily  on  qualitative  controls  and  non-financial  performance  measures 
and  to  a  much  lesser  degree  (if  at  all)  on  quantitative,  financial-performance  measures. 
Contingency  research  on  choice  of  performance  measures  has  yielded  mixed  results,  perhaps 
because  most  of  the  reported  studies  are  based  on  cross-sectional  survey  data,  which  can 
obscure  the  idiosyncrasies  of  firm-level  definitions  and  implementations  of  performance 
measurements  (e.g.,  Anderson  &  Young,  1999;  Chenhall,  2003;  Luft  &  Shields,  2002b).  Enough 
evidence  exists,  however,  to  suggest  that  most  firms  rely  to  some  degree  on  financial 
performance  measures,  and  many  use  both  quantitative  and  qualitative  controls.  In  other  words, 
firms  evidently  have  great  flexibility  to  choose  the  portfolio  of  measures  and  controls  (especially 
when  characterized  as  broad  dichotomies)  that  they  expect  will  work  best  in  their  situations. 

This  equivocal  result  provides  some  motivation  to  search  for  additional  theoretical  explanation 
for  the  choice  of  performance  measures. 2 

Recent  management  control  research  addresses  specific  factors  that  might  explain 
firms’  choices  of  performance  measures  to  achieve  and  maintain  strategic  advantages. 
Laboratory  experiments  (e.g.,  Libby  et  al,  2002;  Lipe  &  Salterio,  2000;  Luft  &  Shields,  2001, 
2002a)  and  surveys  of  management  control  practice  (e.g.,  Ittner  et  al.  2002;  Cavaluzzo  &  Ittner, 
2002;  ittner  &  Larcker,  1998)  have  identified  attributes  of  performance  measures  that  are 


1  See  Chenhall  (2003)  for  a  comprehensive  review  of  management  control  research  and  contingency  theory. 

2  Van  de  Ven  and  Drazin  (1985)  discuss  the  concept  of  equifinality  as  a  serious  impediment  to  progress  in 
understanding  firm  behavior  via  contingency  theory.  We  agree  with  the  concept  of  contingency  theory  that  MCS  and, 
hence,  PMM  are  idiosyncratic  but  not  random. 
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associated  with  use,  usefulness,  and  performance.  When  combined  with  current  resource- 
based  and  systems-based  strategy  theories,  what  emerges  is  a  focus  on  performance 
measures’  attributes  that  supercede  the  popular  financial  vs.  non-financial  dichotomy.  In  all 
cases,  the  literature  cited  in  the  following  subsections  presumes  that  the  organization  seeks  to 
improve  performance  relative  to  its  strategic  goals. 

2. 1  Measures  are  diverse  and  complementary 

Firms’  management  controls  can  benefit  from  greater  diversity  of  performance  measures 
(i.e. ,  operational,  strategic,  financial,  and  non-financial  measures)  if  operational  measures 
reflect  the  current  drivers  of  future  financial  performance  and  are  early  in  the  value  chain  (Ittner 
&  Larcker,  2002).  Milgrom  &  Roberts  (1995)  argue  that,  if  a  diverse  set  of  performance 
measures  is  a  complete  and  complementary  set  (or  system),  using  a  subset  of  measures  leads 
to  inferior  performance.  From  a  similar  systems  perspective,  Warren  (2002)  argues  that 
successful  management  policies  (e.g.,  PMM)  reflect  resource  interdependence  and 
complementarity. 

2.2  Measures  are  objective  and  accurate 

Ijiri  (1967)  long  ago  re-established  the  theoretical  importance  of  (accounting) 
performance  measure  accuracy  and  objectivity.  This  topic  has  not  lost  relevance.3  More 
recently,  Libby  et  al.  (2002)  find  that  experimental  subjects  in  management-control  tasks  rely  on 
performance  measures  that  have  been  verified  by  third  parties,  which  might  create  demand  for 
accurate  and  objective  measures.  Other  studies  have  found  that  low-quality  measurement  is 
associated  with  low  MCS  use  or  impact  (Cavaluzzo  &  Ittner,  2002;  Ittner  &  Larcker,  1998). 
However,  it  is  unclear  ex  ante  if  investing  in  measurements  is  superior  to  measuring  the  wrong 
things  or  the  right  things  poorly,  or  avoiding  unreliable  measures  altogether  (e.g.,  Cavaluzzo  & 
Ittner,  2002;  Gates,  1999).  Objectivity  (or  verifiability)  and  accuracy  (or  error  free)  are 
theoretically  independent  concepts,  yet  are  often  coincident  in  practice  with  reference  to 
performance  measurement. 

2.3  Measures  are  informative 

Performance  measures  that  differentiate  managers  facing  similar,  uncontrollable  factors 
are  informative.  Informative  measures  can  improve  evaluations,  even  if  they  are  not  completely 
controllable  by  managers  (e.g.,  Antle  &  Demski,  1988).  In  particular,  early  value-chain  measures 
can  be  valuable  if  they  are  informative  about  managers’  leading  actions  (Ittner  &  Larcker,  2001) 
in  sufficient  time  to  take  corrective  control  actions. 

2.4  Benefits  outweigh  costs  of  collection 

Monitoring  employee  behavior  through  a  PMM  is  a  costly  activity.  Generating, 
organizing,  and  reporting  performance  information  consume  scarce  company  resources 
(Merchant,  1998;  Simons,  2000).  As  management  accounting  researchers  have  known  since 
the  early  days  of  the  field  (e.g.,  Horngren,  1967),  the  perceived  benefits  of  using  performance 
measures  should  outweigh  the  associated  costs. 


3  Discussions  of  many  current  accounting  and  performance-measurement  issues  can  be  traced  back  to  Ijiri’s  classic 
work. 
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2.5  Measures  reflect  system  causality 

Some  academics  and  consultants  have  prescribed  forms  of  causal  PMM  (e.g.,  Kaplan  & 
Norton,  1996;  Epstein  et  al.,  2000;  Kanji  &  Moura  e  Sa,  2002).  Regardless  of  the  sources  of 
business  models,  causal  relations  among  firms’  multiple  performance  measures  often  are 
neither  specified  nor  measured  well  (Ittner  et  al.,  2002).  Quantifying  cause-and-effect  relations 
between  actions  and  outcomes  at  key  points  in  the  value  chain  could  help  predict  future  effects 
of  current  actions  (e.g.,  Eccles,  1991).  A  functioning  causal  PMM  also  might  free  managers  to 
focus  more  on  strategy  and  evaluation  issues  (e.g.,  Kaplan  &  Norton,  2001)  than  on  information 
processing.  Furthermore,  a  comprehensive,  causal  PMM  might  reduce  the  cognitive  complexity 
of  understanding  and  using  multiple  measures  of  performance  (Luft  &  Shields,  2002a).  Strategy 
theorists  predict  significant  benefits  from  building  causal  models  of  firms’  strategic  resources 
and  capabilities.  Huff  (1 990)  and  Huff  and  Jenkins  (2002)  describe  these  models  as 
knowledge-based,  cognitive  maps,  which  can  connect  and  organize  dispersed  organizational 
knowledge. 

2.6  Measures  communicate  strategy 

Models  such  as  PMM  facilitate  communication,  learning,  and  creation  of  new  knowledge 
and  can  be  the  key  tool  to  building  a  learning  organization  (Huff  &  Jenkins,  2003).  The  right 
performance  measures  align  actions  and  strategy  by  reducing  managers’  financial  myopia 
(McKenzie  &  Schilling,  1998),  and  effectively  communicate  strategy  (Kaplan  &  Norton,  2001; 
Malina  &  Selto,  2001).  Systemic  management  understands  and  exploits  knowledge  of  dynamic 
interrelations  among  resources  and  capabilities.  The  elements  of  a  PMM  are  intended  to  reflect 
the  strategic  use  of  resources  and  deployment  of  efficient  processes  [e.g.,  Sanchez  et  al., 

2002]. 

2.7  Measures  create  incentives  for  improvement 

Using  performance  measures  that  capture  inherent  time  delays  between  certain 
decisions  (e.g.,  investing  in  R&D  and  employee  development)  can  lead  to  improved  incentives 
(e.g.,  Rappaport,  1999;  Cloutier  &  Boehlje,  2002).  Ittner  and  Larcker  (2001)  also  observe  that 
operational  measures,  which  have  good  “line  of  sight,”  can  increase  the  expectancy  of  rewards 
based  on  those  measures  (e.g.,  Green,  1992). 

2.8  Measures  improve  decision-making 

Organization  of  measures  into  distinct  categories  can  affect  decision-making,  perhaps  by 
reflecting  the  structure  of  knowledge  about  the  firm’s  value  chain  (Lipe  &  Salterio,  2000). 
Measures  with  tangible  connections  to  processes  being  managed  also  might  activate  more 
knowledge  and  promote  better  learning  and  decision-making  compared  to  relying  on  financial 
measures  alone  (Luft  &  Shields,  2001 , 2002a).  Huff  and  Jenkins  (2002)  argue  that  models  (e.g., 
PMM)  organize  and  express  the  rationale  of  complex  systems,  which  aid  planning  and 
evaluation  activities.  Furthermore,  such  models  can  represent  micro-  or  macro-levels  of 
knowledge  of  activities,  processes,  and  systems,  thus  aiding  individuals  at  all  levels  of  the 
organization.  PMM  might  improve  decision-making  by  identifying  actions  and  impacts  that 
heretofore  have  been  hidden  by  traditional  measurement  systems  (e.g.,  Huff  &  Jenkins,  2002). 

Management  control  and  strategy  theories  identify  eight  desirable  attributes  of 
performance  measures.  Measures  should  be: 

Al  Diverse  and  complementary 
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A2  Objective  and  accurate 
A3  Informative 

A4  More  beneficial  than  costly 
A5  Causally  related 
A6  Strategic  communication  devices 
A7  Incentives  for  improvement 
A8  Supportive  of  improved  decisions 

3.  Research  Questions 

3.1  RQ1 — Are  measure  attributes  important  considerations  for  performance  measure 
choice? 

The  discussions  in  section  2  provide  extensive  support  for  the  prediction  that  an 
organization  chooses  measures  for  their  PMM  (initially  and  subsequently)  based  on  at  least  the 
eight  attributes  summarized  in  this  study.  The  initial  choice  of  performance  measures  may  be 
based  on  perceived  expected  attributes  since  little  history  exists  upon  which  to  base  the 
assessment.  However,  changes  may  occur  after  experience  proves  the  perceptions  correct  or 
incorrect.  Therefore,  this  study  predicts  that  an  organization  will  delete  previously  chosen  PMM 
measures,  which  after  experience,  are  perceived  to  not  exhibit  these  attributes.  Whether  for 
initial  choice  or  subsequent  change,  the  theoretical  basis  for  attribute  importance  remains  the 
same.  We  expect  that  a  company  chooses  measures  that  it  expects  to  have  the  eight  attributes 
and  keeps  only  those  that  display  the  attributes  in  practice. 

More  specifically,  contingency-based  research  has  shown  that  firm  strategy  can  affect 
the  design  of  PMM  (Chenhall,  2003).  These  studies  typically  use  one  of  several  taxonomies  to 
describe  firms’  strategies,  including  entrepreneurial  or  conservative  (Miller  &  Friesen,  1982), 
prospector,  analyzer,  or  defender  (Miles  &  Snow,  1978),  build,  hold,  or  harvest  (Gupta  & 
Govindarajan,  1984),  and  product  differentiation  or  cost  leadership  (Porter,  1980).  More 
conservative  strategies  include  defender,  harvest,  and  cost  leadership,  while  more 
entrepreneurial  strategies  are  prospector,  build,  and  product  differentiation.  Guilding  (1999) 
found  that  companies  following  a  prospector  or  build  strategy  are  more  likely  to  use  a  broad 
scope  of  PMM  information.  Govindarajan  and  Gupta  (1985)  found  that  companies  with  a  build 
strategy  are  more  inclined  to  use  subjective  controls  than  those  pursuing  a  harvest  strategy. 
Several  studies  find  that  firms  following  a  more  conservative  strategy  place  more  emphasis  on 
cost  control  than  those  following  a  more  entrepreneurial  strategy  (Chenhall  &  Morris,  1995; 

Dent,  1990;  Simons,  1987).  Bouwens  and  Abernethy  (2000)  found  that  firms  going  through  a 
strategic  change  process,  typically  categorized  as  an  entrepreneurial  activity,  place  more 
importance  on  integrated  PMM  information.  Finally,  Abernethy  and  Brownell  (1999)  found  that 
hospitals  following  a  prospector  strategy  focused  more  attention  on  dialogue,  communication 
and  learning.  No  prior  strategic-fit  work  was  found  leading  to  a  strategy-based  preference  for 
attributes  of  informativeness  and  incentives  for  improvement.  Table  1  maps  the  prior  strategic- 
fit  research  to  the  eight  performance  measure  attributes  in  Table  1 . 
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Table  1.  Framework  for  Performance-Measure  Attributes 


Strategy 


Performance- 
Measure  Attribute 

Entrepreneurial  Firm 
(prospector,  build,  product 
differentiation, 
entrepreneur) 

Conservative  Firm 
(defender,  harvest, 
cost  leader, 
conservative) 

Study 

A1  Diverse  and 
complementary 

X 

Guilding,  1999 

A2  Objective  and 
accurate 

X 

Govindarajan  &  Gupta, 
1985 

A3  Informative 

? 

? 

No  studies  of 
informativeness  related 
to  strategy 

A4  Benefit  exceeds 
cost 

X 

Chenhall  &  Morris,  1995 
Dent,  1990 

Simons,  1987 

A5  Causally  related 

X 

Bouwens  &  Abernethy, 
2000 

A6  Strategic 

communication 

device 

X 

Abernethy  &  Brownell, 
1999 

A7  Incentive  for 
improvement 

? 

? 

No  studies  of  incentives 
related  to  strategy 

A8  Improved 
decision-making 

X 

Abernethy  &  Brownell, 
1999 

Although  each  of  the  eight  attributes  is  desirable  (ceteris  paribus),  strategic-fit  research 
implies  that  some  attributes  may  be  more  relevant  than  others  depending  upon  company 
strategy.  It  is  expected  that  firms  following  an  entrepreneurial  strategy  will  be  more  likely  to 
prefer  measures  that  are  diverse  and  complementary  (A1),  causally  related  (A5),  strategic 
communication  devices  (A6),  and  improve  decision-making  (A8).  It  is  also  expected  that  firms 
following  a  more  conservative  strategy  will  tend  to  prefer  performance  measures  that  are 
objective  and  accurate  (A2)  and  more  beneficial  than  costly  (A4). 


3.3  RQ3 — Does  the  importance  of  attributes  for  design  and  use  differ  according  to  firm 
strategy? 

Although  all  eight  are  desirable  attributes,  a  company  may  deem  some  more  relevant 
than  others  when  designing  or  using  measures  in  a  PMM.  The  exploratory  third  question 
explores  subjective  classifications  of  the  eight  attributes  as  being  primarily  being  driven  by  either 
design  or  use  concerns.  Performance  measure  attributes  reflecting  primarily  design 
considerations  are  diverse  and  complementary  (A1 ),  objective  and  accurate  (A2),  informative 
(A3),  more  beneficial  than  costly  (A4),  and  causally  related  (A5).  Although  all  performance 
measures  are  at  least  minimally  designed,  we  conjecture  that  some  are  selected  as  means  to 
achieve  specific  ends.  They  can  be  used  to  communicate  strategy  (A6),  provide  incentives  for 
improvement  (A7),  and  improve  decision-making  (A8).  We  test  whether  the  importance  differs 
according  to  this  partition  of  design  and  use  attributes. 

Because  it  is  possible  that  a  measure  can  be  useful  and  kept  in  a  PMM  even  if  it  does 
not  exhibit  all  attributes,  we  also  explore  whether  an  organization  trades-off  one  attribute  for 
another.  The  strategic-fit  literature  cited  previously  can  lead  one  to  expect  tradeoffs  when 
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measures’  attributes  are  not  consistently  high.  For  example,  a  company  following  conservative 
strategies,  given  costly  measurement,  might  (not)  choose  or  retain  a  measure  that  is  high  (low) 
on  accuracy  and  objectivity  but  low  (high)  on  strategic  communication.  A  more  entrepreneurial 
firm,  on  the  other  hand,  might  (not)  choose  or  retain  a  measure  that  is  high  (low)  on  improved 
decision  making  but  low  (high)  on  the  benefit-cost  ratio.  Although  one  generally  might  expect 
these  tradeoffs,  they  would  be  quite  difficult  to  predict  for  a  specific  firm  without  having  more 
precise  attribute  measurements.  Thus,  the  fourth  research  question  also  is  exploratory. 

Figure  1  summarizes  the  four  research  questions. 


Figure  1.  Research  Questions 


4.  Research  Site 

The  research  takes  place  within  a  single  firm  to  benefit  from  close  investigation  of  (1)  a 
PMM  developed  by  the  firm,  not  imposed  by  the  researchers  (or  external  consultants),  (2) 
measures  of  performance  relevant  to  that  firm,  not  generic  measures  that  might  or  might  not 
apply  to  the  firm,  and  (3)  access  to  multiple  levels  of  managers  to  enrich  the  understanding  of 
the  origins  and  uses  of  the  PMM.  Thus,  this  study  offers  some  advantages  compared  to  cross- 
sectional  analysis  at  the  firm  level,  particularly  given  the  difficulty  of  comparably  describing 
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performance  measurement  in  many  firms  simultaneously  [e.g.,  Luft  and  Shields,  2002b].  This 
study  had  access  to  company  documents,  archival  data,  and  employees  at  various  levels. 

This  study  focuses  on  a  PMM  developed  by  a  US  FORTUNE  500  equipment 
manufacturer  for  its  distribution  channel.4  The  company  employs  over  25,000  people  in  its 
domestic  and  foreign  operations.  Competition  is  from  similarly  large  domestic  and  international 
firms.  The  company  sells  its  major  equipment  assemblies  through  OEM  contracts  and 
independent,  exclusive  distributors,  who  also  sell  repair  parts  and  maintenance  services.  The 
distribution  system  is  the  company’s  primary  contact  with  retail  customers.  Each  distributor 
operates  within  an  assigned  geographic  area.  The  distributors  may  not  compete  with  each  other 
or  sell  competitors’  products  or  services.  While  the  distributorships  are  independent  entities, 
most  are  owned  by  individuals  with  prior  company  experience. 

4.1  Company  Strategy 

Per  archival  documents,  the  company  follows  five  strategic  initiatives,  which  are: 

1 .  Demonstrate  a  comparative  advantage  in  each  of  our  markets  worldwide, 
measured  in  product  performance,  economic  value  to  the  customer  and  all 
aspects  of  customer  support. 

2.  Achieve  an  average  return  on  equity  of  at  least  x%  over  economic  cycles  in  order 
to  afford  the  investment  required  to  sustain  a  comparative  advantage  in  each 
market  into  the  future,  and  afford  investment  in  new  business. 

3.  Grow  in  order  to  provide  superior  total  return  to  our  shareholders  over  time. 

4.  Demonstrate  our  commitment  to  help  improve  the  community  in  which  we 
operate  and  be  a  responsible  citizen  of  society. 

5.  Attract,  train,  challenge  and  fully  utilize  people  at  all  levels  in  order  to  achieve 
these  objectives. 

The  company’s  strategy  can  be  characterized  along  the  lines  of  conservative,  defender, 
harvest,  and  cost  leader.  Competitive  advantage  is  gained  by  aggressively  preventing 
competitors  from  entering  the  primary  product  market.  Product  performance,  quality  and 
efficiency  have  been  the  company’s  means  to  success. 

Historically,  the  company  has  managed  its  distributors  top-down  and  strictly  by  the 
impact  on  the  company’s  “bottom-line.”  However,  recent  changes  in  the  marketplace  have 
affected  the  company’s  approach  to  measuring  distributor  performance.  Because  both 
domestic  and  foreign  competitors  now  have  similar  products,  competition  for  market  share  has 
shifted  to  customer  service.  In  response  to  this  increased  competition,  the  company  revised  its 
distributor  strategy  to  focus  attention  and  resources  on  improving  customer  service  and 
customer  satisfaction. 

The  company’s  top-down  approach  to  distributor  management  is  consistent  with  the 
theory  that  firms  following  a  conservative  strategy  rely  more  heavily  on  centralized  control 
systems  and  on  feed-forward  control  (Langfield-Smith,  1997).  Its  main  distribution  goal  is 


4  The  company  has  created  PMM  for  other  elements  of  its  value  chain  but  gave  access  to  only  the  distribution  PMM. 
Other  PMMs  reportedly  are  being  used  successfully. 
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improving  the  primary  market  share  for  its  products.  Profitability  at  the  distributor  level  was 
important  (particularly  to  distributors)  but  less  important  than  primary  market  share  because 
most  of  the  company’s  profit  derived  from  sales  in  this  market.  Again,  this  supports  the 
classification  of  the  company  strategy  as  more  conservative.  Conservative  companies  tend  to 
use  control  systems  that  signal  the  need  for  innovation  by  highlighting  substantial  drops  in 
market  share  and  declining  profitability  (Miller  &  Freisen,  1982). 

4.2  Distributor  Performance  Model 

A  major  step  in  the  company’s  management  of  its  distribution  system  was  to  develop  a 
strategically  oriented  PMM  for  its  31  North  American  distributorships  that  communicates  the 
new  customer  focus,  guides  distributors,  and  provides  additional  bases  for  evaluation.  This 
study  will  refer  to  the  company’s  distributor  performance  model  as  the  DPM.5 

According  to  company  documents,  the  purposes  of  the  DPM  are  to: 

•  Highlight  areas  within  distributorships  that  need  improvement  to  enhance  customer 
relations, 

•  Provide  an  objective  set  of  criteria,  consistent  with  the  company’s  strategic  initiatives,  to 
guide  and  measure  total  distributor  performance, 

•  Be  used  as  the  starting  point  for  the  three-year  distributor  contract  renewal  process,  and 

•  Be  used  for  comparing  and  ranking  distributors  and,  perhaps,  for  performance-based 
compensation. 

The  DPM  was  developed  internally  by  company  employees — without  the  aid  of  external 
consultants.  As  was  customary  in  this  company,  a  top-down  approach  was  used.  The  DPM 
designers  created  the  initial  DPM  with  selective  input  from  distributor  personnel,  although  the 
designers  had  the  final  say.  The  DPM  was  designed  to  focus  on  outcomes  that  the  company 
felt  were  important  for  the  distributors  to  accomplish  in  order  to  meet  company  goals,  not 
necessarily  distributor  goals.  Therefore,  the  company’s  key  success  factors  do  not  perfectly 
mirror  those  of  its  distributors.6 

The  data  for  this  study  come  from  interviews  with  managers,  from  company  documents 
and  from  archival  performance  data.  During  the  second  quarter  of  1999,  sixteen  interviews 
were  conducted:  nine  with  distributor-owners  and  seven  with  DPM  designers  and 
administrators.  Distributors,  designer  and  administrator  views  were  sought  in  order  to  have  a 
360  degree  view  of  the  DPM  process.7  The  length  of  the  telephone  interview  was  determined 
by  how  much  the  interviewee  had  to  say;  all  lasted  between  45  and  75  minutes.  The 
researchers  asked  each  interviewee  the  following  open  questions:8 

1.  In  your  own  words,  what  is  the  DPM? 


5 

Company  employees  refer  to  the  DPM  as  a  “balanced  scorecard,”  but  the  term  has  acquired  a  generic  label  that 
might  obscure  the  unique  characteristics  of  this  PMM. 

6  For  a  discussion  of  the  conflict  resulting  from  this  initial  mismatch,  see  Malina  and  Selto  (2001 ). 

7  For  a  complete  discussion  of  the  sampling  technique,  see  Malina  and  Selto  (2001 ). 

Q 

These  interviews  also  were  used  to  explore  communication  effectiveness  in  Malina  and  Selto  (2001).  The  present 
study  reflects  an  independent  use  and  extension  of  these  qualitative  data. 
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2.  What  do  you  think  is  the  objective  of  the  DPM? 

3.  What  are  the  nine  measures  that  distributors  report  really  measuring? 

4.  What  are  the  measures  that  are  filled  out  by  the  company  really  measuring? 

5.  How  do  the  measures  that  distributors  report  relate  to  the  company’s  measures? 

6.  Do  the  measures  help  you  in  any  way? 

7.  Are  there  any  benefits  from  the  DPM  itself? 

8.  Do  you  have  any  (other)  recommendations  for  improving  the  DPM? 

Note  that  the  questions  do  not  directly  prompt  respondents  to  discuss  the  factors  that 
are  predicted  to  determine  choices  or  changes  in  either  measures  or  weights.  The  questions 
were  purposefully  generic  (providing  a  framework  for  discussing  DPM  measures)  but  not  directly 
asking  about  choice  or  change.  Interviewees  freely  revealed  these  factors  during  the 
interviews.  Their  unprompted  responses  were  used  to  support  or  deny  the  predictions. 9 

Company  documents  provided  archival  background  for  the  structure  of  the  DPM  and 
quantitative  data  for  each  quarterly  DPM  from  the  first  quarter  of  1998  to  the  fourth  quarter  of 
2001.  The  interviews  were  analyzed  using  qualitative  data  software  (Atlas. ti).  Qualitative  data 
software  is  used  to  systematically  code  the  qualitative  data.  A  predetermined  set  of  codes  was 
used  to  identify  portions  of  interview  text  referring  to  a  choice  or  a  change  in  DPM  measures 
according  to  each  of  the  eight  attributes  developed  from  the  literature.  Although  all  respondents 
had  multiple,  coded  comments,  the  coding  procedure  gave  each  respondent  only  a  single  code 
for  each  intersection  of  performance  measure  and  attribute.  Thus,  if  one  interviewee,  for 
example,  offered  a  paragraph  describing  the  accuracy  of  a  specific  measure,  this  response 
generated  a  single  set  of  codes — one  code  for  the  measure  and  one  for  the  attribute.  For  each 
coded  comment  regarding  an  attribute,  an  additional  code  was  attached  regarding  whether  the 
measure  was  dropped  from  the  DPM  or  remained  on  the  DPM  during  the  time  period  tested.10 
This  restrictive  approach  to  coding  qualitative  data  is  designed  to  illustrate  or  test  theory. 11  A 
complete  list  of  codes  used  in  this  research  is  shown  in  Table  2. 


9  Perhaps  respondents  would  have  made  more  comments  specific  to  the  hypothesized  attributes  if  we  had  posed 
direct  questions.  However,  we  wanted  to  avoid  responses  that  were  artifacts  of  leading  questions.  We  do  not  have 
the  luxury  of  a  parallel  case  study  where  we  could  have  asked  leading  questions  to  compare  with  our  results 

10  Two  researchers  independently  coded  the  qualitative  data.  The  average  coding  reliability  between  the  two  coders 

was  87%,  which  falls  above  the  minimum  norm  of  80  percent  coding  reliability  (Miles  &  Huberman,  1994).  Coding 
discrepancies  were  reconciled  by  consensus.  The  consensus  coding  supports  the  reported  qualitative  analyses.  For 
a  complete  discussion  of  insuring  coding  reliability,  see  Malina  and  Selto  (2001). 

1 1 

Miles  and  Huberman  (1994)  provide  extensive  explanations  of  alternative  coding  methods. 
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Table  2.  Coding  Scheme 

Code 

1.1  A1+:  Diverse  and  complementary 

1 .2  A1-:  Not  diverse  and  complementary 

2.1  A2+:  Objective  and  accurate 

2.2  A2-:  Not  objective  and  accurate 

3.1  A3+:  Informative 

3.2  A3-:  Not  informative 

4.1  A4+:  Strategic  communication  devices 

4.2  A4-:  Not  strategic  communication  devices 

5.1  A5+:  Incentives  for  improvement 

5.2  A5-:  No  incentives  for  improvement 

6.1  A6+:  Supportive  of  improved  decisions 

6.2  A6-:  Not  supportive  of  improved  decisions 

7.1  A7+:  More  beneficial  than  costly 

7.2  A7-:  Not  more  beneficial  than  costly 

8.1  A8+:  Causally  related 

8.2  A8-:  Not  causally  related 

9.1  Measure  dropped  from  DPM 

9.2  Measure  kept  on  DPM 


5.  Data  Analysis 

Section  5  is  divided  into  two  sub-sections.  The  first  sub-section  describes  evidence 
regarding  the  initial  choice  of  performance  measures.  The  second  sub-section  uses  both 
qualitative  and  quantitative  analyses  to  evaluate  interview  evidence  related  to  the  eight 
attributes  regarding  changes  of  performance  measures. 

Throughout  the  four  years  covered  by  this  research,  the  company  made  many  changes 
to  the  DPM.  Weightings  changed,  measures  were  added,  and  measures  were  dropped.  The 
initial  DPM  (implemented  in  the  first  quarter  of  1998)  and  the  revised  DPM  are  shown  in  table  3. 
Table  4  contains  DPM  measure  definitions.  The  most  obvious  change  in  the  revised  DPM  is  a 
reduction  in  the  number  of  measures  included.  The  first  DPM  contained  29  measures  while  the 
most  recent  has  14.  Also,  all  three  of  the  “people”  measures  (performance  reviews,  industry 
involvement,  and  training)  were  dropped  from  the  DPM.  Of  the  fifteen  measures  dropped  from 
the  initial  DPM,  fourteen  were  eliminated  at  the  end  of  1999. 
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Table  3.  Comparison  of  Distributor  Performance  Models 

Q1  1998  to  Q4  2001 

Q1  1998 

Q4  2001 

Measure 

Weight 

Measure 

Weight 

Customer  Satisfaction 

3.0 

Customer  Satisfaction 

10 

Market  Share  -  Traditional  Market 

27.0 

Market  Share  -  New  Market  1 

3.0 

Market  Share  -  New  Market  2 

3.0 

Dealer  Survey 

2.0 

Dealer  Survey 

10 

Parts  Fill  Rate 

3.0 

Parts  Fill  Rate 

5 

Service  Cycle  Time 

10.0 

CME 

3.0 

Best  Practices 

1.0 

PBIT  as  %  of  Sales 

4.0 

PBIT  as  %  of  Sales 

10 

Cash  Flow  as  %  of  Sales 

2.0 

Parts  Inventory  Turns 

2.0 

Parts  Inventory  Turns 

5 

Whole  Goods  Inventory  Turns 

2.0 

Whole  Goods  Inventory  Turns 

5 

Days  Sales  Outstanding 

2.0 

Days  Sales  Outstanding 

5 

Service  Utilization 

2.0 

Service  Utilization 

5 

Parts  Sales  Growth  (traditional) 

2.0 

Parts  Sales  Growth  (traditional) 

5 

Service  Sales  Growth  (traditional) 

2.0 

Service  Sales  Growth  (traditional) 

5 

New  Market  1  Sales  Growth 

2.0 

New  Market  1  Sales  Growth 

10 

New  Market  2  Sales  Growth 

1.0 

New  Market  2  Sales  Growth 

10 

Other  Sales  Growth 

2.0 

Environmental  Assessment 

2.0 

Safety 

2.0 

Safety 

5 

Performance  Reviews 

1.0 

Industry  Involvement 

1.0 

Training 

2.0 

Warranty  Audit 

8.0 

Building  Condition 

3.0 

Policy  Dollars 

2.0 

Days  to  Fin.  Statement  Submission 

1.0 

Image 

10 

DPM  Total  Weight 

100 

DPM  Total  Weight 

100 
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Measure 

Customer  Satisfaction 
Market  Share  Measures 
Market  Share  - 
Traditional  Market 
Market  Share  -  New 
Market  1 

Market  Share  -  New 
Market  2 
Dealer  Survey 
Parts  Fill  Rate 
Service  Cycle  Time 

CME 

Best  Practices 

PBIT  as  %  of  Sales 
Cash  Flow  as  %  of  Sales 
Parts  Inventory  Turns 
Whole  Goods  Inventory 
Turns 

Days  Sales  Outstanding 
Service  Utilization 
Parts  Sales  Growth 

Service  Sales  Growth 

New  Market  1  Sales 
Growth 

New  Market  2  Sales 
Growth 

Other  Sales  Growth 

Environmental 

Assessment 

Safety 

People  Measures 

Performance  Reviews 
Industry  Involvement 
Training 
Warranty  Audit 
Building  Condition 
Policy  Dollars 

Days  to  Fin.  Statement 

Submission 

Image 


Table  4^DPM  Measure  Definitions _ 

Definition 

Score  on  customer  satisfaction  event  card 

Monthly  percentage  of  distributor  share  of  total  traditional  market 

Monthly  percentage  of  distributor  share  of  total  new  market  1 

Monthly  percentage  of  distributor  share  of  total  new  market  2 

Annual  survey  of  dealer  satisfaction  with  distribution  system 
Percentage  of  parts  orders  filled  within  24  hours 
Percent  of  service  repairs  diagnosed  within  one  hour  and 
completed  within  six  hours 

Audit  of  distributor  use  of  Company  Marketing  Excellence  system 

Number  of  best  practices  submitted  or  number  of  best  practices 

implemented  from  data  base  in  the  last  12  month  period 

PBIT  as  a  percentage  of  sales 

Cash  flow  from  operations  as  a  percentage  of  sales 

Parts  cost  of  sales  divided  by  average  parts  inventory 

Whole  goods  cost  of  sales  divided  by  average  parts  inventory 

Average  trade  receivable  balance  divided  by  sales  per  day 
Technician  hours  billed  divided  by  hours  available  to  be  billed 
12  quarter  rolling  regression  over  the  most  recent  12  quarters  parts 
sales  growth 

12  quarter  rolling  regression  over  the  most  recent  12  quarters  non¬ 
warranty  sales  growth 

12  quarter  rolling  regression  over  the  most  recent  12  quarters  new 
market  1  sales  growth 

12  quarter  rolling  regression  over  the  most  recent  12  quarters  new 
market  2  sales  growth 

12  quarter  rolling  regression  over  the  most  recent  12  quarters  other 
sales  growth 

If  distributor  has  completed  an  environmental  assessment  per 
Company  guidelines. 

Lost-time  accidents  per  200,000  hours  worked 

Random  sample  of  percent  of  employees  evaluated  annually 

Evidence  of  membership  in  industry  and  trade  associations 

Percent  of  hours  spent  on  training  per  hours  worked 

Percent  of  compliance  of  warranty  repairs 

Company  determined  rating  of  distributor  properties 

Comparison  of  actual  versus  planned  expenditure  on  non-warranty 

repairs 

Number  of  days  from  close  of  accounting  period  to  submission  of 

financial  statement  to  Company 

(Not  implemented  at  the  time  of  this  research) 
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Other  obvious  changes  in  the  DPM  are  the  weights  assigned  to  measures.  In  the  first 
DPM,  weightings  ranged  from  1.0  to  27  points.  As  the  company  shortened  the  DPM,  remaining 
measures  ultimately  were  given  either  a  5-  or  10-point  weight.  Table  5  chronicles  the  revisions 
in  the  weights  assigned  to  the  measures  dropped  from  the  DPM  over  the  study  period. 


Table  5.  Revised  Performance  Weights  of  Dropped  Measures 
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The  following  subsections  discuss  evidence  from  the  interviews  that  indicates  support  or 
lack  of  support  for  predicted  reasons  behind  the  company’s  initial  choices  of  all  measures  and 
changes  to  them. 
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5. 1 1nitial  choice  of  performance  measures 

Company  designers  and  managers  volunteered  eleven  comments  regarding  how  and 
why  the  initial  measures  appeared  on  the  DPM.  The  most  cited  reason  (5  comments,  or  almost 
half  of  total  comments)  for  including  the  initial  measures  was  that  they  are  objective  and 
accurate.  A  company  employee  stated: 

The  objective  of  the  DPM  is  to  have  an  objective,  documented,  factual 
measurement  system  rather  than  a  subjective  one. 

Several  interviewees  commented  that  the  initial  measures  were  chosen  for  strategic 
communication  and  causality.  Three  comments  were  made  that  the  initial  measures  were 
designed  to  communicate  the  corporate  strategy  down  to  the  distributor  level.  A  DPM  designer 
stated: 


The  measures  were  chosen  to  dovetail  the  core  objectives. 

The  measures  were  also  chosen  with  cause-and-effect  relations  in  mind.  Three 
comments  were  made  that,  by  design,  the  initial  DPM  measures  were  intuitively  related  to  each 
other.  A  DPM  designer  commented: 

The  theory  of  how  the  measures  tie  together  is  good.  There  are  logical 

correlations. 

The  coded  interviews  show  objectivity  and  accuracy,  strategic  communication  devices 
and  causality  as  the  primary  bases  for  initial  choices  of  DPM  measures.  The  company’s 
conservative  tradition  can  be  seen  in  its  emphasis  on  objectivity  and  accuracy  in  performance 
measures.  However,  the  change  in  the  approach  to  measuring  distributor  performance  appears 
as  emphasis  was  also  placed  on  the  attributes  of  strategic  communication  and  causality. 

5.2  Analysis  of  RQ1 — Are  measure  attributes  important  considerations  for  performance 
measure  choice? 

5.2.1  Qualitative  analysis  of  RQ1 

The  following  subsections  describe  the  results  of  the  qualitative  analysis  of  changes  in 
measures  summarized  in  Table  6.  Table  6  presents  the  distributions  of  codes,  by  positive  or 
negative  form  of  the  attributes,  for  measures  dropped  from  the  DPM  and  for  those  retained  on 
the  DPM.  Columns  record  the  frequency  of  comments  associated  with  dropped  and  kept 
measures.  Rows  record  the  frequency  of  comments  associated  with  positive  and  negative 
perceptions.  The  code  frequencies  in  columns  1  and  2  are  meant  to  reassure  the  reader  that 
the  authors  did  not  selectively  focus  on  unrepresentative  comments  for  the  analyses  that  follow. 
Column  3  shows  the  number  of  respondents  comprising  the  frequency  of  comment  and  is 
meant  to  reassure  the  reader  that  one  or  two  interviewees  did  not  dominate  the  results. 
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Table  6.  Changes  in  DPM  Measures  Code  Frequency 


Measure 

Dropped 

Measure 

Retained 

Number  of 
Respondents 

Design  Attributes 

A1+:  Diverse  and  complementary 

0 

3 

3 

A1-:  Not  Diverse  and  complementary 

11 

4 

8 

Total 

11 

7 

A2+:  Objective  and  accurate 

14 

12 

13 

A2-:  Not  Objective  and  accurate 

46 

19 

13 

Total 

60 

31 

A3+:  Informative 

0 

0 

0 

A3-:  Not  Informative 

0 

0 

0 

Total 

0 

0 

A4+:  Benefits  exceed  costs 

3 

2 

3 

A4-:  Costs  exceed  benefits 

17 

0 

9 

Total 

20 

2 

A5+:  Causally  related 

13 

23 

13 

A5-:  Not  Causally  related 

4 

0 

4 

Total 

17 

23 

Use  Attributes 

A6+:  Strategic  communication 

17 

6 

13 

devices 

A6-:  Not  Strategic  communication 

1 

0 

1 

devices 

Total 

18 

6 

A7+:  Incentives  for  improvement 

18 

6 

14 

A7-:  No  Incentives  for  improvement 

2 

3 

5 

Total 

20 

9 

A8+:  Supportive  of  improved 

20 

7 

9 

decisions 

A8-:  Not  Supportive  of  improved 

3 

0 

3 

decisions 

Total 

23 

7 

Total  Positive  Comments 

85 

59 

Total  Negative  Comments 

84 

26 

Total  Comments 

169 

85 

5.2. 1.1  Diverse  and  complementary  (A1).  The  DPM  was  intended  to  be  a  broad  set  of 
measures  that  encompasses  the  activities  that  distributors  must  manage  well  to  create  a 
successful  distributorship.  The  original  intent  was  to  reinforce  that  distributors  needed  to  look  at 
all  aspects  of  the  business,  from  hiring  and  training  employees  to  winning  market  share. 
Distributors  recognized  the  complementarity  of  the  DPM’s  diverse  measures.  For  example, 

We  can’t  have  good  results  without  good  people  who  are  trained.  We 
can’t  have  good  customer  satisfaction  without  good  fill  rate  and  cycle  time 
results.  They  aii  relate  to  sales  growth  and  profitability.  I  see  them  definitely 
[as  complementary]. 
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Most  distributors,  however,  complained  about  the  lack  of  diversity  in  original  market- 
share  measures.  Distributors  felt  that  traditional  market-share  measures  were  overrepresented 
and  over-weighted  on  the  DPM.  Distributors  face  diverse  opportunities.  Some  make  most  of 
their  profit  from  the  traditional,  mature  market,  but  many  can  earn  more  from  the  new,  growing 
markets.  Although  diverse  market-share  measures  were  included  on  the  initial  DPM,  only  the 
traditional  measure  carried  significant  weight. 

They  aren’t  measuring  anything  but  [traditional]  business,  [sarcastically:] 

If  they  are  only  interested  in  about  one-third  of  their  business,  then  it’s  good. 

The  measures  chosen  to  be  on  the  DPM  were  to  be  diverse  and  complementary.  As 
noted  above,  this  was  not  perceived  always  to  be  the  case.  Of  the  18  comments  coded  to  this 
attribute,  15  supported  the  negative  form  that  measures  are  not  diverse  and  complementary. 

5.2. 1.2  Objective  and  accurate  ( A2 ).  Objectivity,  accuracy,  reliability,  and  auditability 
appear  to  have  been  the  company’s  and  distributors’  primary  concerns  for  DPM  measures. 
Interviewees  commented  on  this  attribute  most  frequently,  with  91  comments  in  total. 

The  traditional  market-share  measure  dominated  the  original  DPM  in  part  because  it  was 
highly  objective  and  accurate. 

We  know  every  single  [product  in  the  traditional  market]  that  gets  sold  to 
the  tenth  of  a  percentage  point. 

In  contrast,  the  new  market-share  measures  were  perceived  to  be  less  accurate  and 
less  objective. 

How  we  would  measure  [new]  market  share  is  strictly  information  we 
would  generate  ourselves. 

Almost  every  distributor  commented  on  certain  measures  being  inaccurate  or  subject  to 
manipulation.  In  general,  the  distributors  felt  that  the  people  measures  were  not  well  defined  or 
verifiable.  Six  of  the  nine  distributors  mentioned  that  these  measures  involved  some  guessing 
and  that  there  was  no  rigorous  audit  process  in  place  to  verify  the  data  reported.  Early  in  the  life 
of  the  DPM,  the  distributors  manually  collected  and  reported  service  cycle  time,  which  the 
company  dropped  after  a  few  years.  Six  of  the  nine  distributors  commented  on  the  lack  of 
accuracy  in  reporting  this  measure. 

I’m  going  to  make  that  number  look  as  good  as  I  can  without  outright  lying 
or  cheating. 

DPM  administrators  also  were  aware  that  the  measure  might  not  be  accurate. 

/  wouldn’t  put  a  lot  of  stock  in  either  the  [service  cycle-time  measures]  we 
got  before  or  the  ones  on  this  scorecard. 

Nearly  every  measure  dropped  from  the  DPM  lacked  perceived  objectivity  and/or 
accuracy.  The  exception  was  the  traditional  market-share  measure  which  was  replaced  with  the 
almost  equally  objective  sales-growth  measure.  The  majority  of  the  comments  supported  the 
negative  form  of  the  attribute  that  measures  are  neither  objective  nor  accurate. 
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5. 2. 1.3  Informative  (A3).  Relatively  few  respondents  indicated  that  informativeness  was 
an  important  design  criterion.  The  company  uses  the  DPM  to  compare,  benchmark,  and  rank 
distributorships  and  as  a  stimulus  to  peer  communication.  Each  distributor  receives  its  own 
report  and  its  relative  numerical  ranking  (e.g.,  7th  out  of  31).  To  promote  information  exchange 
and  competition;  the  names  of  distributors  achieving  top  ratings  are  posted  on  the  company’s 
intranet  for  all  distributors  to  see. 

Individuals  can  determine  their  performance  against  someone  else  in  the 
corporation  performing  the  same  function,  so  they  would  know  how  effective 
they  were  at  accomplishing  their  goals  and  their  tasks. 

Several  distributors  recognized  the  influence  of  peer  pressure  on  their  behavior. 

Anytime  you  publish  a  report  and  there  are  31  entities  being  measured 
using  the  same  metric,  you  create  competition.  We  are  competitive,  so  it 
matters  what  rank  you  are.  Even  if  no  one  looks  at  the  rank,  I  want  to  be  #1. 

Qualitative  evidence  exists  to  support  the  importance  of  DPM  informativeness,  but  not 
necessarily  at  the  individual-measure  level.  That  is,  the  company  appears  to  use  the  overall 
DPM  score  to  rank  distributor  performance  more  than  the  scores  on  individual  measures. 
Therefore,  this  attribute  cannot  be  analyzed  at  the  individual-measure  level  and  will  be  withheld 
from  the  quantitative  analyses  that  follow. 

5.2. 1.4  Costs  versus  benefit  (A4).  The  company  perceived  a  number  of  DPM  measures 
to  be  misleading  or  unreliable  and  also  perceived  the  costs  of  resolving  disputes  about  these 
subjective  measures  to  be  greater  than  the  benefits  derived.  Seventeen  comments  stated  that 
the  cost  to  compile  or  the  time  spent  resolving  disputes  outweighed  the  benefits  received  from 
collecting  the  measure.  The  performance  evaluation  and  service  cycle-time  measures  in 
particular  were  seen  as  consuming  too  many  resources. 

To  do  [performance  evaluations]  four  times  a  year  for  everyone  in  the 
company,  there  aren’t  enough  hours  in  the  day.  They  need  to  find  something 
more  appropriate  for  technicians  and  clerks. 

It’s  just  adding  time  and  effort  to  report  [service  cycle  time]  to  the 
company.  If  we’re  doing  this,  we’re  not  doing  things  for  the  customer. 

Agreement  existed  among  the  interviewees  that  the  cost  of  some  measures  exceeded 
their  benefits,  providing  support  for  the  negative  form  of  this  attribute. 

5.2. 1.5  Causally  related  (A5).  The  causal  nature  of  the  DPM  was  intended  by  designers 
and  generally  perceived  by  distributors.  The  DPM  measures  were  grouped  and  saliently 
displayed  to  guide  decisions  and  recognize  the  company’s  beliefs  about  the  relations  among 
measures  within  the  group.  Representative  comments  from  distributors  reflect  the  intuition  and 
general  belief  in  the  DPM’s  causal  relations. 

A  lot  of  business  tends  to  run  with  financial  and  market  share  measures. 

Those  are  pretty  crude  handles.  You  have  to  get  underneath  to  things  like 
quality  and  cycle  times,  and  softer  things  like  employee  development.  That’s 
where  the  leverage  of  the  business  is.  The  others  are  the  results  of  what 
you’ve  done. 
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My  intuition  is  that  the  growth  goals  help  drive  market  levels,  but  we  don’t 
know  the  quantitative  consequences.  Some  measures  are  more  tightly 
correlated,  but  I  don’t  have  a  sense  of  which  of  these  are  the  key  ones  that 
seem  to  have  the  most  leverage  compared  to  others. 

Ninety  percent  of  the  comments  coded  to  this  attribute  supported  the  positive  form  that 
DPM  measures  reflect  causality. 

5.2. 1.6  Strategic  communication  devices  (A6).  At  the  time  of  the  DPM  rollout,  the 
company  also  launched  its  new  customer-oriented  objective.  The  original  DPM  measures 
covered  the  value-chain  activities  that  the  company  felt  were  critical  to  creating  a  distributorship 
that  competes  successfully  on  customer  satisfaction — the  keystone  of  the  company’s  strategy  to 
sustain  market  share.  DPM  designers  observed: 

Those  [measures  and  categories]  are  right  in  line  with  the  strategic 
initiatives,  aligned  with  our  customer  and  quality  objectives.  We  try  to  dovetail 
any  plans  the  company  has  into  those  objectives. 

All  but  one  comment  were  in  support  of  the  positive  form  of  A6  that  measures  are 
strategic  communication  devices.  Distributors  and  designers  all  agreed  that  DPM  measures 
helped  communicate  corporate  strategy  to  the  distribution  system. 

5.2. 1.7  Incentives  for  improvement  (A7).  The  comments  regarding  incentive  effects  of 
DPM  measures  were  generally  in  support  of  the  positive  form  of  this  attribute  that  measures 
provided  incentives  for  improvement.  Service  cycle  time,  best  practices,  and  people  measures 
were  all  cited  as  creating  incentives  for  distributors  to  improve  performance. 

Best  practice  is  a  positive  measure  because  it  forces  you  to  take  a  look  at 
other  ways  to  run  your  business. 

Comments  regarding  market  share  were  split.  Distributors  commented  that  the 
emphasis  on  traditional  market  was  discouraging  to  those  distributors  who  saw  more  growth 
potential  in  the  newer  markets. 

If  you  did  poorly  on  [the  traditional  market  share]  and  well  on  [the  new 
market  share  measures],  then  you’re  still  viewed  as  a  poor  distributor. 

The  ultimate  control  over  distributor  behavior  has  been  the  three-year  contract  review, 
when  the  company  evaluates  distributor  performance  and  either  renews  or  terminates  the 
relationship.  The  DPM  in  total  is  intended  to  provide  a  constructive  and  evaluative  structure  and 
to  be  an  objective  basis  for  the  contract  review.  Designers  stated: 

The  only  incentive  is  losing  the  distributorship  and  [the  DPM]  is  the 
centerpiece  of  the  contract  review. 

The  stick  might  be  more  evident  than  any  carrot,  because  as  one  top  manager  candidly 

stated, 


It  [the  DPM]  helps  when  it  comes  time  to  terminate  a  distributor.  If  you’ve 
set  up  goals  and  a  distributor  has  failed  to  achieve  them,  you  can  get  around 
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sales  representation  and  franchise  laws  for  wrongful  termination.  It’s  not  the 
primary  objective,  but  it’s  a  benefit. 

Nearly  every  comment  supported  the  positive  form  of  A7  that  measures  provide 
incentives  for  improvement. 

5.2. 1.8  Supportive  of  improved  decisions  (A8).  The  company  chose  measures  for  the 
DPM  to  reflect  the  activities  and  types  of  financial  and  non-financial  performance  believed  to  be 
effective  to  achieve  increased  company  profitability.  Most  distributors  agreed: 

The  elements  and  the  structure  [of  the  DPM]  are  outstanding,  and  they 
have  a  lot  of  potential  to  help  us  all  improve.  [...]!  grew  up  working  for  a  CPA 
and  he  ingrained  in  me  that,  if  you  can’t  measure  it,  you  can’t  improve  it. 

Several  of  the  measures  on  the  initial  DPM  that  were  subsequently  eliminated  also  were 
perceived  to  be  helpful  for  decision-making.  In  regard  to  training,  one  distributor  mentioned: 

As  a  result  of  the  measure,  we  more  formally  measure  this.  I  like  that 
because  it  lets  me  see  which  areas  of  the  company  are  doing  training  and 
which  aren’t.  I  can  chat  with  those  not  doing  training.  They  get  so  tied  up 
doing  other  work  that  it’s  hard  to  put  time  aside. 

The  service  cycle-time  measure,  when  it  was  part  of  the  DPM,  did  affect  decision¬ 
making  in  the  service  process  at  most  distributorships.  Distributors  often  mentioned  that  it 
helped  them  to  redesign  processes  and  re-schedule  work. 

I  wasn’t  an  advocate  [of  the  service  cycle-time  measure]  at  the  start,  but 
now  I  am.  It  tells  us  to  quickly  figure  out  what’s  wrong  so  we  can  make  an 
intelligent  statement  to  the  customer,  so  they  can  say  go  ahead  or  not.  We 
have  been  able  to  flow  more  work  through  our  show  by  getting  the  quick, 
easy  stuff  through  the  shop.  [. . .]  It’s  helping  us. 

All  but  three  comments  supported  DPM  measures  as  being  helpful  in  day-to-day 
managing  and  decision-making,  therefore  providing  support  for  the  positive  form  of  the  attribute. 

5.2.2  Quantitative  analysis  of  RQ1 — Are  measure  attributes  important  considerations  for 
performance  measure  choice? 

If  theories  of  performance  measurement  choice  are  descriptive,  comments  from 
interviewees  should  tend  to  reflect  favorable  perceptions  of  the  attributes  of  retained  measures 
(the  positive  form  of  the  attributes),  whereas  comments  should  reflect  unfavorable  perceptions 
of  the  attributes  of  dropped  measures  (the  negative  form  of  the  attributes).  Table  7  presents  the 
distributions  of  codes,  by  positive  or  negative  form  of  the  attributes,  for  measures  dropped  from 
the  DPM  and  for  those  retained  on  the  DPM.  A  chi-square  test  compares  the  observed 
distribution  of  comments  to  the  expected  distribution,  which  assumes  that  dropping  or  keeping  a 
measure  is  unrelated  to  positive  or  negative  forms  of  the  attributes. 


ACQUISITION  RESEARCH:  THE  FOUNDATION  FOR  INNOVATION 


-  198- 


Table  7.  Research  Question  1: 

Positive  versus  Negative  Comments 
Analysis  of  Attribute  Codes  and  Measure  Retention 


Actual  Distribution  No  Effect  Distribution 

Measure  Measure  Measure  Measure 


Dropped 

Retained 

Dropped 

Retained 

Chi- 

square 

P- 

value 

< 

Positive 

85 

59 

96 

48 

8.42 

0.01 

comments 

Negative 

84 

26 

73 

37 

comments 

Total 

169 

85 

comments 


The  Chi-square  test  of  RQ1  indicates  that  the  comment  pattern  is  not  random,  but  is 
significantly  different  (p  <  0.01)  from  the  expected  (no  effect)  distribution.  This  result  reinforces 
the  fact  that  the  qualitative  results  that  measure  attributes  are  important  considerations  for 
dropping  or  keeping  DPM  measures.  Retained  measures  are  more  likely  to  have  positive  forms 
of  the  attributes,  and  dropped  measures  are  more  likely  to  have  negative  forms  of  the  attributes. 
This  result,  which  is  consistent  with  the  theory  presented  in  section  2,  demonstrates  that 
measure  attributes  are  important  considerations  for  performance-measure  choice. 

Research  question  2  asks  if  organizational  strategy  is  related  to  the  relative  importance 
placed  on  performance-measure  attributes.  As  described  in  the  previous  section,  the  company 
itself  follows  a  rather  conservative  strategy  and  is  expected  to  prefer  conservative  performance 
measures.  Table  8  presents  the  distributions  of  codes,  by  positive  or  negative  form,  of  attributes 
expected  to  be  consistent  with  a  conservative  strategy  and  of  those  expected  to  be  consistent 
with  an  entrepreneurial  strategy. 

The  Chi-square  tests  of  the  conservative  and  entrepreneurial  strategy  attributes  indicate 
that  the  patterns  are  not  random,  and  significant  differences  exist  between  the  observed  and 
expected  (no  effect)  distributions  (p  <  0.025).  For  the  conservative  attributes  test,  the  largest 
impact  on  the  Chi-square  statistic  results  from  a  larger-than-expected  number  of  positive 
comments  about  retained  measures.  Conversely,  for  the  entrepreneurial  attributes  test,  the 
largest  impact  on  the  Chi-square  statistics  stems  from  a  smaller-than-expected  number  of 
negative  comments  that  are  associated  with  retained  measures.  These  results  indicate  that  both 
conservative  and  entrepreneurial  attributes  have  been  important  to  the  company’s  choices,  in 
subtly  different  ways.  Measures  consistent  with  a  conservative  strategy  were  retained  if  they 
were  positively  perceived,  while  measures  consistent  with  an  entrepreneurial  strategy  appear  to 
be  retained  if  comments  were  not  overly  negative. 
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Table  8.  Research  Question  2: 

Conservative  versus  Entrepreneurial  Strategy  Attributes 
_ Analysis  of  Attribute  Codes  and  Measure  Retention _ 

Actual  Distribution  No  Effect  Distribution 

Measure  Measure  Measure  Measure  Chi- 

Dropped  Retained  Dropped  Retained  square  value 

< 

Conservative 

Attributes 


Positive  comments  - 

17 

14c 

22 

9C 

5.26 

0.025 

Conservative  Attributes 

a 

Negative  comments 

63 

19 

58 

24 

-  Conservative 

Attributes 

Total  comments 

80 

33 

Entrepreneurial 

Attributes 


Positive  comments  - 

50 

39 

55 

34 

5.40 

0.025 

Entrepreneurial 

Attributes  b 

Negative  comments 
-  Entrepreneurial 
Attributes 

19 

4° 

14 

9° 

Total  comments 

69 

43 

a  Attributes  predicted  to  be  most  applicable  for  a  conservative  strategy  are  A2  (objective  and 
accurate)  and  A4  (cost  versus  benefit). 

b  Attributes  predicted  to  be  most  applicable  for  an  entrepreneurial  strategy  are  A1  (diverse  and 
complementary),  A5  (causally  related),  A6  (strategic  communication  device),  and  A8  (improved  decision¬ 
making). 

c  Distribution  value  that  contributes  the  most  to  the  Chi-square  statistic.  The  5-comment  variance 
represents  a  55.5%  difference  between  the  actual  and  expected  distribution. 

5.4  Quantitative  analysis  of  RQ3 — Does  the  importance  of  attributes  for  design  and  use 
differ  according  to  firm  strategy? 

Table  9  presents  the  distributions  of  comment  codes,  by  positive  or  negative  form,  of  the 
attributes  divided  into  design  and  use.  This  exploratory  Chi-square  test  indicates  that  the 
overall  pattern  is  not  random,  but  is  significantly  different  (p  <  0.001)  from  the  expected  (no 
effect)  distribution.  The  largest  impact  on  the  Chi-square  statistic  comes  from  positive 
comments  about  design  attributes.  Next,  strategy  is  introduced  to  investigate  whether  firm 
strategy  affects  the  importance  of  design  and  use  attributes.  Since  attributes  associated  with  a 
more  conservative  strategy  are  exclusively  design  attributes,  the  Chi-square  test  is  run  solely  for 
design  attributes.  Thus,  this  test  is  identical  to  that  reported  in  Table  8;  the  firm  appears  to  have 
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considered  positive  design  attributes  identified  with  a  more  conservative  strategy  when  deciding 
to  retain  a  performance  measure  (p  <  0.025). 

The  final  Chi-square  test  indicates  that  the  firm  also  considered  design  attributes 
associated  with  an  entrepreneurial  strategy  to  be  important  considerations  (p  <  0.001).  Thus, 
design  attributes  of  measures  appear  to  dominate  usage  attributes,  regardless  of  the  strategic 
orientation  of  measures. 


Table  9.  Research  Question  3: 

Design  versus  Use  Attributes 
Analysis  of  Attribute  Codes  and  Measure  Retention 


Actual  Distribution 

No  Effect  Distribution 

Measure 

Measure 

Measure 

Measure 

Dropped 

Retained 

Dropped 

Retained 

Chi- 

p-value 

square 

< 

Overall 

Positive  comments  - 

30 

40d 

47 

23d 

24.61 

0.001 

Design  attributes  a 

Negative  comments  - 
Design  attributes 

78 

23 

67 

34 

Positive  and  negative 
comments  -  Use  attributes13 

61 

22 

55 

28 

Total  comments 

169 

85 

Conservative 

Positive  comments  - 

17 

14e 

22 

9e 

5.26 

0.025 

Design  attributes 

Negative  comments  - 

63 

19 

58 

24 

Design  attributes 

Total  comments 

80 

33 

Entrepreneurial 

Positive  comments  - 

13 

26f 

24 

15f 

20.28 

0.001 

Design  attributes 

Negative  comments  - 
Design  attributes 

15 

4 

12 

7 

Positive  and  negative 
comments  -  Use  attributes c 

41 

13 

33 

21 

Total  comments 

69 

43 

a  Design  attributes  include  A1  (diverse  and  complementary),  A2  (objective  and  accurate),  A4  (cost 
versus  benefit),  and  A5  (causally  related) 

b  Use  attributes  include  A6  (strategic  communication  devices),  A7  (incentive  for  improvement), 
and  A8  (improved  decision-making).  Cells  with  small  expected  counts  (n  <5)  have  been  collapsed. 

cUse  attributes  associated  with  an  entrepreneurial  strategy  include  A6  (strategic  communication 
devices)  and  A8  (improved  decision-making).  Cells  with  small  expected  counts  (n  <5)  have  been 
collapsed. 

d  Distribution  value  that  contributes  the  most  to  the  Chi-square  statistic.  The  1 7-comment 
variance  represents  a  73.9%  difference  between  the  actual  and  expected  distribution. 
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e  Distribution  value  that  contributes  the  most  to  the  Chi-square  statistic.  The  5-comment  variance 
represents  a  55.5%  difference  between  the  actual  and  expected  distribution. 

f  Distribution  value  that  contributes  the  most  to  the  Chi-square  statistic.  The  1 1-comment 
variance  represents  a  73.3%  difference  between  the  actual  and  expected  distribution. 

To  explore  whether  the  company  trades-off  some  attributes  for  others,  comments 
contrary  to  expectations  for  dropped  measures  were  investigated  further,  as  shown  in  Table  10. 
The  first  column  of  numbers  in  Table  10  reproduces  comment  frequencies  shown  in  Table  6  for 
entrepreneurial  use  attributes.  The  company  dropped  measures  that  are  strongly  perceived  as 
strategic  communication  devices  (A6+,  17  comments)  and  supportive  of  improved  decisions 
(A8+,  20  comments),  when  straightforward  predictions  indicate  that  the  measures  should  have 
been  retained. 


Table  10.  Research  Question  4: 
Trade-Off  of  Individual  Attributes 


Comments  in  Unexpected  Direction_ Associated  Unfavorable  Comments 


Measures  are: 

Favorable  Comments 
on  Dropped  Measures 

A1- 

A2- 

A4- 

A5- 

A6- 

A8- 

None 

A6+:  Strategic 
communication  devices 

17 

1 

4 

2 

0 

1 

9 

A8+:  Supportive  of 
improved  decisions 

20 

0 

6 

9 

0 

0 

4 

Total 

37 

1 

10 

11 

0 

0 

1 

13 

Proportions  of  total 
comments 

1.000 

.027 

.270 

.297 

.000 

.000 

.027 

.350 

A1-:  Not  diverse  and  complementary 

A2-:  Not  objective  and  accurate 

A4-:  Not  more  beneficial  than  costly 

A5-:  Not  causally  related 

A6-:  Not  strategic  communication  devices 

A8-:  Not  supportive  of  improved  decisions 

To  determine  whether  the  company  trades  off  these  apparently  beneficial  attributes,  we 
investigated  comments  referring  to  other  attributes  within  the  same  section  of  text.  A  qualitative 
database  query  using  Atlas.ti  identified  co-occurrence  or  associations  between  coded  sections 
of  text.12  The  columns  of  Table  10  labeled  A1-  to  A8-  present  the  number  of  times  that  favorable 
comments  in  the  first  column  are  associated  with  negative  comments  for  each  of  the  other 
attributes.  For  example,  interviewees  said  that  the  people-related  performance  measures, 
among  others,  help  distributors  make  improved  decisions  (i.e.,  support  for  A8+).  However,  the 
people  measures  (among  others)  were  dropped  from  the  DPM. 


Co-occurrence  or  proximity  rules  include  coded  quotations  of  one  type  that  enclose,  are  enclosed  by,  overlap,  are 
overlapped  by,  preceded  by  one-line,  or  followed  by  one-line  coded  quotations  of  another  type.  See  Appendix  B  of 
Malina  and  Selto  (2001 )  for  a  complete  discussion  of  finding  associations  among  codes  using  qualitative  software. 
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The  Atlas. ti  query  tool  found  co-occurring  comments  (27  percent  of  the  total)  stating  that 
the  dropped  measures,  which  ought  to  support  improved  decisions,  are  not  objective  and 
accurate  (A2-).  This  indicates  that  the  DPM  attribute  of  improved  decision-making  is 
subordinate  to  the  attribute  of  objectivity  and  accuracy.  More  co-occurring  comments  (27.9 
percent)  stated  that  dropped  measures  were  not  cost-beneficial.  Almost  no  other  types  of  co¬ 
occurring  negative  comments  appeared.  When  the  company  unexpectedly  dropped  measures, 
it  consistently  made  choices  consistent  with  the  proposition  that  the  attributes  of  objectivity  and 
accuracy  (A2)  and  benefits  versus  costs  (A4)  were  more  important  than  attributes  of  strategic 
communication  (A6)  and  support  of  improved  decisions  (A8). 

6.  Conclusions  and  Discussion 

This  study  focuses  on  the  efforts  of  one  large  firm  to  model  drivers  of  its  distribution 
performance.  It  complements  recent  research  and  offers  an  analytic  generalization  to  an 
emerging  theory  of  performance  measurement  and  management  control.  This  study  contributes 
to  analytic  generalization  about  performance  measurement  by  empirically  examining  choices 
and  revisions  of  diverse  performance  measures  in  a  live  setting.  The  results  of  this  qualitative 
investigation  augment  previous  survey,  experimental,  and  normative  studies  that  collectively  are 
refining  a  theory  of  performance  measurement. 

6. 1  Theory-based  analyses 

Management  control  and  strategy  theories  identify  at  least  eight  desirable  attributes  of 
performance  measures.  Measures  should  be  diverse  and  complementary,  objective  and 
accurate,  informative,  more  beneficial  than  costly,  causally  related,  strategic  communication 
devices,  incentives  for  improvement,  and  supportive  of  improved  decisions.  Analysis  of 
interviews  with  company  PMM  designers  and  users  find  that,  taken  as  a  whole,  these  attributes 
are  relevant  to  decisions  to  delete  or  retain  performance  measures.  This  study  finds  that  the 
studied  organization’s  choices  of  performance  measures  appear  to  be  strongly  influenced  by 
measures’  attributes,  consistent  with  theory. 

Further  analysis  yields  two  results  related  to  the  attributes’  relative  importance  for 
retaining  or  dropping  performance  measures.  First,  the  relative  importance  of  performance 
measure  attributes  in  decisions  to  drop  or  retain  measures  differs  subtly  according  to  firm 
strategy.  Positive  attributes  of  conservative-strategy  measures  and  relative  lack  of  negative 
attributes  of  entrepreneurial-oriented  measures  are  associated  with  the  measures  retained  on 
the  PMM.  Because  the  company  itself  followed  a  rather  conservative  strategy,  we  expected 
conservative-strategy  measures  to  be  dropped  only  if  they  had  an  abundance  of  negative 
attributes.  This  appeared  to  be  the  case.  Likewise,  because  of  the  top-down  nature  of 
distribution  management,  we  expected  that  the  company  would  keep  only  the  entrepreneurial- 
strategy  measures  (urged  by  entrepreneurial  distributors)  that  had  a  relative  abundance  of 
positive  attributes.  However,  in  this  case,  entrepreneurial  measures  tended  to  be  kept  as  long 
as  they  were  perceived  to  not  have  excessive  negative  attributes.  This  result  appears  somewhat 
inconsistent  with  our  interpretations  of  prior  contingency  theory  strategic-fit  research  on 
attributes. 

We  identify  three  possible  reasons  for  this  mild  contradiction.  First,  prior  research 
investigates  strategy  fit  at  the  PMM  level,  not  at  the  individual-measure  level.  Our  predictions  of 
which  attributes  are  more  applicable  for  the  two  broad  definitions  of  firm  strategy  are  based  on 
the  applicability  of  these  PMM-level  results  to  measure-level  analysis.  The  measurement-level 
of  analysis  may  be  contributing  to  the  unexpected  result. 
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Second,  the  classification  of  firm  strategy  does  not  lend  itself  to  a  clear-cut  conservative 
or  entrepreneurial  dichotomy.  As  cautioned  by  Chenhall  (2003),  the  extant  typologies  might  be 
overly  simplistic  for  modern,  dynamic  firms.  Most  aspects  of  the  company  strategy  reflect  a 
conservative  strategy;  yet,  not  all  follow  neatly  in  line.  For  example,  the  recent  change  to  focus 
on  customer  satisfaction  is  more  in  line  with  an  entrepreneurial,  differentiation  strategy  than  a 
conservative  strategy. 

Finally,  and  related  to  the  difficulty  of  neatly  categorizing  a  company’s  strategy,  two 
related  entities  are  being  studied.  The  company  might  follow  a  conservative  strategy  while  the 
distributors,  although  still  fitting  mostly  into  a  conservative-strategy  classification,  have  distinct 
entrepreneurial  tendencies.  The  company’s  narrow  performance  focus  had  caused 
considerable  tension  between  management  and  regional  distributors  who  saw  more  profitable 
opportunities  in  the  company’s  secondary  markets.  Unlike  the  company,  many  distributors’ 
success  stems  from  tapping  the  less-established  and  growing  secondary  markets.  This 
strategic  tension  played  out  in  the  evolution  of  a  PMM  for  the  distribution  channel  and  may  be 
contributing  to  the  subtle  difference  in  criteria  for  keeping  or  dropping  performance  measures.  In 
other  words,  distributors,  because  they  have  direct  impacts  on  customers  and  company  sales, 
might  have  more  influence  on  PMM  design  than  we  anticipated. 


6.2  Exploratory  analyses 

Attributes  that  we  associated  with  design  appear  to  be  more  influential  than  those 
associated  with  use.  This  finding  led  to  an  investigation  of  trade-offs  among  attributes. 
Although  all  of  the  attributes  might  be  desirable,  the  two  attributes  that  appear  most  associated 
with  keeping  or  dropping  individual  DPM  measures  are  (a)  the  objectivity  and  accuracy  of 
measures  and  (b)  the  cost  versus  benefit  balance  of  measurement.  A  measure  might  promote 
good  decision-making  and  communicate  strategy,  for  example,  but  the  company  dropped  it  if 
experience  showed  it  could  not  be  measured  inexpensively,  accurately  and  objectively.  Trade¬ 
offs  also  permitted  a  more  detailed  analysis  of  the  effect  of  strategy  on  PMM  choice.  At  an 
aggregate  level,  only  a  subtle  difference  was  found  in  the  importance  of  attributes  consistent 
with  either  a  conservative  versus  an  entrepreneurial  strategy.  However,  at  the  individual- 
attribute  level,  we  find  that  objectivity  and  accuracy  (as  well  as  cost  versus  benefit — both 
consistent  with  the  company’s  conservative  strategy)  accompany  changes  in  DPM  measures. 


6.3  Discussion 

The  trade-off  of  attributes  can  be  seen  clearly  in  the  treatment  of  market-share 
measures.  As  mentioned  previously,  the  company  initially  emphasized  the  importance  of  the 
traditional,  primary  market  share.  Primary  market  share  was  important  to  corporate  strategy, 
but  the  company  also  had  the  ability  to  measure  primary  market  share  extremely  accurately  and 
objectively.  Many  distributors  complained  that  the  traditional  market  did  not  afford  them  growth 
opportunities  and  that  more  weight  should  be  attached  to  new  market  shares.  Unfortunately, 
new  market  shares  are  difficult  to  measure  accurately  and  objectively.  The  company  ultimately 
eliminated  all  the  market-share  measures  from  the  DPM.  The  company  decided  to  move  the 
weight  assigned  to  primary  market  share  to  sales-growth  measures.  Measurement  of  sales 
growth  is  easily  obtained,  and  relatively  accurate  and  objective  for  both  the  traditional  market 
and  the  new  markets,  although  sales  growth  does  not  speak  directly  to  the  company’s  strategy 
to  grow  market  share.  However,  the  addition  of  sales-growth  measures  retained  diversity  among 
measures  and  recognized  distributors’  entrepreneurial  opportunities. 
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Apparently  this  company,  with  its  long  history  of  conservative,  bottom-line  financial 
management,  could  not  tolerate  subjectivity  in  measurement  or  high  costs  of  improved 
information,  nor  could  it  tolerate  the  costly  disputes  that  inevitably  follow  the  use  of  subjective 
measures  for  performance  evaluations  and  contracting.  This  finding  has  implications  for  the  use 
of  performance  measures  (e.g.,  some  non-financial  measures)  that  cannot  be  measured  or 
audited  as  objectively  or  accurately  as  financial  measures.  For  example,  many  organizations 
currently  measure  employee  capabilities;  some  might  use  these  measures  in  performance 
evaluations.  A  full  understanding  of  the  “softness”  of  such  measures  might  preclude  non¬ 
productive  disputes  that  can  arise  if  they  are  used  to  evaluate  and  reward  performance, 
particularly  in  firms  that  are  accustomed  to  using  hard,  financial  measures.  The  creative  task 
facing  PMM  designers  is  to  find  measures  that  are:  diverse  and  complementary,  informative, 
strategic  communication  devices,  incentives  for  improvement,  supportive  of  improved  decisions, 
causally  related,  but  are  also  objective  and  accurate  and  not  too  costly.  Clearly,  this  is  a 
challenging  task. 

6.4  Future  research 

Future  research  might  replicate  this  study  in  similar  or  dissimilar  firms  or  to  a  large 
sample  of  diverse  firms  to  obtain  useful  contrasts  or  further  support  for  the  theory  of 
performance-measurement  models.  Future  research  questions  include: 

•  What  are  the  measurement  tradeoffs  among  measure  attributes  that  might  be  necessary 
to  implement  feasible  PMM? 

•  Do  all  firms  (or  only  those  with  a  conservative  strategy)  stress  cost  of  measurement, 
accuracy  and  objectivity  of  performance  measures  above  all  other  attributes? 

•  Is  complementarity  an  issue  beyond  initial  design?  Do  returns  or  tradeoffs  to  scale  and 
scope  of  measurement  exist? 

•  What  are  the  performance  opportunity  costs  of  trading-off  performance-measure 
attributes? 

6.5  Epilogue 

Three  years  after  the  introduction  of  the  DPM,  as  reported  in  Malina  and  Selto  (2001), 
distributors’  measured  financial  performance  improved  significantly.  The  DPM  is  alive  and  well 
as  of  May  2005,  and  it  is  used  for  managing  and  evaluating  distributors  and  distributorships. 

Top  management  commitment  to  using  and  refining  the  DPM  was  and  continues  to  be  strong. 
Distributors  continue  to  adapt  their  behavior  to  the  DPM’s  guidance,  and  the  DPM  has 
undergone  several  more  changes  since  the  completion  of  this  research.  An  example  of 
adaptation  is  that  all  distributors  now  meet  the  DPM’s  challenging  safety  goal,  despite  many 
early  complaints  that  it  was  an  impossible  standard  (see  Malina  &  Selto,  2001 ).  Changes  to  the 
DPM  include  dropping  formal  reporting  of  profit  (PBIT/S)  because  of  unforeseen  claims  by  labor 
unions  for  a  greater  share  of  profits,  and  inclusion  of  several  new  measures  that  had  been 
proposed  earlier  but  not  yet  developed.  The  DPM  continues  to  be  the  center  point  of  the 
contract-renewal  process,  and  its  increasing  objectivity  and  relevance  has  added  to  its 
acceptance.  The  DPM  has  become  an  integral  part  of  management  control  at  this  company  and 
undoubtedly  will  continue  to  evolve. 
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